The significant disruptions caused by the COVID-19 pandemic led many social scientists to turn toward online survey experimentation for empirical research. Generalizing from the experiments conducted during this period of persistent crisis may be challenging, both because of changes to the demographic characteristics of online survey respondents, and how they respond to treatments. We investigate this using 32 replication experiments conducted across 13 national surveys of U.S. residents between March and July. We find that replication estimates closely match those from original studies in nearly all cases, and estimates from COVID-specific framing experiments are statistically indistinguishable from direct replications. These findings mitigate concerns about the temporal validity of online research during this period. Consistent with an apparent decline in attentiveness among survey respondents, however, effect sizes are often smaller than those from the pre-COVID period. This suggests additional caution is warranted in the design, analysis, and interpretation of experiments conducted during the COVID-19 pandemic.

About the presenter:

I’m a Postdoctoral Fellow in Law and Social Science in the Justice Collaboratory at Yale Law School. My research addresses pressing questions at the intersection of politics, policy, and intergroup relations, with a focus on causal inference in survey, lab, and field settings. This work draws upon diverse theoretical frameworks from political science, psychology, and sociology to understand how experiences with government affect individual’s beliefs, preferences, and behaviors. I have used this approach to ask questions like: How can police departments change officer behavior to foster public trust and cooperation, particularly within minority communities where relations are characterized by longstanding conflict and distrust?

I received a PhD in Political Science (conferred May 2020) and MA in Statistics and Data Science from Yale University. My dissertation, “Experiments on Legitimacy and Intergroup Relations: Policing, Trust, and Prejudice in the United States” won the James G. March Award for Outstanding Dissertation in Political Science from Yale, and the Best Dissertation Award from the American Political Science Association’s Experiments Section. My work is published or forthcoming in Proceedings of the National Academy of Science, American Political Science Review, and Journal of Politics, among other outlets.

Zoom Details

Meeting Link
Meeting Password: 1234